Particular embodiments generally relate to a framework to allow parallel processing of tasks in an adaptable manner.
When there is a problem to be solved, a user may write a program for a specific type of machine. The machine can execute the program and perform analysis on certain chunks of data. However, the data processed is limited by resources of the machine. Also, multiple machines may be configured to process the chunks of data in parallel. However, the program needs to be written specially for the multiple machines to process the data. Writing a program for multiple machines takes an experienced programmer who is familiar with the configuration of the machines. This becomes complicated and is not useful for a user who does not understand how to write programs for processing on multiple machines.